Also fix CometArrayExpressionSuite compilation by qualifying the Spark
udf() call, which was shadowed by the new org.apache.comet.udf package.

Implements a DataFusion PhysicalExpr that evaluates child expressions,
exports the results as Arrow FFI arrays, calls
CometUdfBridge.evaluate() via JNI, and imports the output array.
Adds datafusion-comet-jni-bridge as a dependency of the spark-expr crate.

… is true

… wording

…UDF class via context classloader

Wrap the JNI body in try/finally so input ValueVectors and the result vector
are always closed, even when the UDF or arrow export throws. Resolve the
CometUDF class through the thread context classloader so user-supplied UDF
jars (added via spark.jars) are visible from the bridge.

…ns fall back to Spark

When routing RLike through the JVM UDF, reject Literal(null) and patterns
that fail Pattern.compile during planning. Both cases now produce withInfo
+ None, letting Spark evaluate the expression instead of crashing the
executor task with PatternSyntaxException or NullPointerException.

Make comet_udf_bridge an Option in JVMClasses so a missing
org.apache.comet.udf.CometUdfBridge class (e.g. shading dropped
org.apache.comet.udf.*) no longer crashes executor JVM init. The
JVM-UDF dispatch path returns a clear ExecutionError when the bridge
is unavailable. Also clarify the FFI lifetime contract on the result
import.

Replace string literals "rust"/"java" used for the regexp engine selector
with named constants on CometConf. Tighten CometRLike.getSupportLevel so
it only reports Compatible(None) when the pattern is a Literal, matching
the actual constraint enforced by the convert path.

Literal-folded children no longer get expanded to batch-row count before
crossing JNI; ColumnarValue::Scalar is materialized at length 1, avoiding
an O(rows) copy of values that never vary across the batch. Document the
contract on CometUDF: scalar inputs arrive as length-1 vectors, vector
inputs at the batch row count, and the result must match the longest
input.

…suite

Adds a master switch (default false) for the experimental JVM UDF framework
so the Java regex engine cannot be activated without an explicit opt-in. With
engine=java but jvmUdf.enabled=false, the six regex serdes return Unsupported
with a message naming the master switch instead of silently using either path.

Also extends Incompatible with optedInBy: Option[String] so a config (e.g. an
engine selector) can serve as a per-expression incompatibility opt-in. Existing
allowIncompatible flags continue to work; optedInBy is OR'd into the gating
check in QueryPlanSerde. No existing serde uses optedInBy yet — this lays the
foundation for the config simplification discussed in apache#4310.

Drop redundant interpolators in COMET_REGEXP_ENGINE doc string and remove
the redundant CometConf self-import in CometStringExpressionSuite to
satisfy scalafix. Switch existing rlike/regexp_replace tests to opt in via
COMET_REGEXP_ENGINE=rust now that the engine selector is the gate for the
Rust path, and reformat regex.md via prettier.

Resolve conflicts in pr_build_{linux,macos}.yml by integrating both the
new codegen-suite additions from main and the CometRegExpJvmSuite from
the PR, dropping the obsolete standalone "sql" matrix entry that main
folded into the "spark" matrix.

Resolve CometConf.scala by retaining both COMET_JVM_UDF_ENABLED /
COMET_REGEXP_ENGINE from the PR and COMET_SCALA_UDF_CODEGEN_ENABLED
from main. The follow-up refactor drops COMET_JVM_UDF_ENABLED in favor
of COMET_SCALA_UDF_CODEGEN_ENABLED.

…of hand-written UDFs

Replace the six hand-written `RegExp*UDF` / `StringSplitUDF` JVM UDF
implementations with the Arrow-direct codegen dispatcher introduced in
PR apache#4417 (`CometScalaUDF.emitJvmCodegenDispatch`). The dispatcher
Janino-compiles Spark's own `doGenCode` for the expression, so the
regex family inherits Spark-identical semantics with no per-expression
glue code.

Changes:

- Delete `spark/src/main/scala/org/apache/comet/udf/RegExp*UDF.scala`
  and `StringSplitUDF.scala`. Their behavior is now provided by
  Spark's `doGenCode` running inside the dispatcher.
- Rewrite the regex serdes in `strings.scala`. Expressions with no
  native Rust path (`RegExpExtract`, `RegExpExtractAll`, `RegExpInStr`)
  share a new `CometRegexpCodegenOnly` base; expressions with a native
  path (`RLike`, `RegExpReplace`, `StringSplit`) keep an explicit
  route table where the JVM arm now delegates to
  `CometScalaUDF.emitJvmCodegenDispatch`.
- Drop the `spark.comet.jvmUdf.enabled` config. The codegen dispatcher
  already has its own master switch
  (`spark.comet.exec.scalaUDF.codegen.enabled`); gating the regex
  family on the same flag avoids two flags for the same path.
  `spark.comet.exec.regexp.engine` keeps the `java`/`rust` selector
  semantics, and `engine=java` now requires the codegen flag.
- Revert the native Rust additions in `jvm_udf/mod.rs` and
  `jni-bridge/src/lib.rs`. The codegen dispatcher constructs Arrow
  output fields JVM-side via `CometBatchKernelCodegenOutput.toFfiArrowField`,
  so the list-vector field-name normalization cast is unnecessary.
- Update `CometRegExpJvmSuite`, `CometRegExpBenchmark`, the regex SQL
  test fixtures, and the regex compatibility doc to reflect the new
  gating.

Test plan:
- `CometRegExpJvmSuite`: 45/45 pass (covers all six regex expressions
  through the codegen dispatcher).
- `CometSqlFileTestSuite`: 289/289 pass.
- `CometStringExpressionSuite`: 33/33 pass.
- `CometCodegenSuite`: 60/60 pass.
- `cargo clippy --all-targets --workspace -- -D warnings`: clean.

Flip spark.comet.exec.scalaUDF.codegen.enabled back to false and restore
the experimental, disabled-by-default language across the regex/codegen
docs and CometConf strings. With this default, the regex family (java
engine path) and the DateFormat dispatcher fall back to Spark unless the
user explicitly opts in.

This keeps the engine=rust JVM-dispatcher fallthrough behavior introduced
separately on this branch; only the codegen-enabled-by-default change is
reverted.

Remove the remaining experimental/disabled-by-default framing from
regex.md so the Java engine reads as a normal, supported regex engine
gated behind spark.comet.exec.scalaUDF.codegen.enabled.